The present invention relates to the correction of planar focus in an electronic still or video camera.
Desktop flat bed scanners are very common in office imaging applications. Although these are relatively inexpensive and work well, a disadvantage is that these invariably take up a significant amount of desk space, which is always at a premium.
Digital camera products are becoming common in many areas of still and motion photography, and as a result are becoming ever less expensive. However such cameras are still used almost exclusively for photography of people or places, and have yet to be adapted for use in office imaging applications. This is because electronic cameras, which normally employ two dimensional CCD arrays, have insufficient resolution to image a complete A4 page at 300 dots per inch, the minimum that is conventionally believed necessary for reasonably high quality reproduction. Whilst larger CCDs are available in electronic cameras, they are much too expensive for a mass-market office imaging product.
A camera mounted over the desk and looking directly downwards could image documents on the desktop, avoiding this permanent waste of space. In order to obtain sufficient resolution with the limited number of available pixels, the camera would need to be directly above the document to be imaged, so that all of the document was in best focus. This, however, is inconvenient, requiring either that a user lean directly over the document or that a frame is provided over the desk on which the camera would be mounted. An overhead camera could obstruct a user""s headroom or be inadvertently knocked.
Alternatively, if the camera were not directly above the document but, say, held or mounted at near an edge of the desktop, then not all of the document would be in focus simultaneously because the document would not be at right angles to the optical axis of the camera. Owing to the loss of resolution due to the camera""s limited depth of field, only a part of the document would be captured with sufficient resolution in one frame. This is inconvenient, because a user or some type of mechanical actuator would then have to manually pan and tilt the camera to take a number of overlapping pictures, each with a magnification high enough to obtain sufficient resolution, and subsequently xe2x80x9cstitchxe2x80x9d these pictures together using software.
There is also the additional problem that the image would be distorted owing to the oblique viewing angle, an effect, referred to herein as xe2x80x9ckeystone distortionxe2x80x9d. Although such perspective distortion can readily be rectified using well-known image processing techniques, this does result in non-optimal resolution over the portions of the image where a printed character is spread over fewer pixels.
Conventional approaches to addressing these problems suffer from other limitations. Greater depth of field can be provided by reducing the camera aperture but this lowers the amount of light at the image plane, which raises noise levels. A greater exposure time will not work with hand-held photography, owing to camera shake. Active or electronic image stabilizers add to cost, and are not fully effective at eliminating the effect of camera shake. A frame and mechanical mounting to hold and pan/tilt the camera can eliminate camera shake, but with a significant penalty in terms of mechanical complexity and cost. Even with a small aperture, or longer shutter time, it is difficult to obtain the necessary depth of field to operate down to an angle of less than 45 degrees, as may be desirable with an electronic camera held or mounted at the edge of a desktop.
For many years, photographers have understood that under certain circumstances as defined by the Scheimpflug condition, it is possible simultaneously to focus on several things at different distances from the camera, providing that all the objects of interest lie on a flat plane. When the Scheimpflug condition is satisfied, the object plane, image plane, and a plane passing through the lens, all intersect along a line. Whilst this condition is necessary for correct focus, it is not, on its own, sufficient. Setting up a camera to focus on a tilted object plane has up to now been regarded as requiring a tripod mounted large format view camera and the skill of a professional photographer.
Professional photographers using such large format cameras can, with practice and an intuitive understanding of three dimensional geometry, achieve the Scheimpflug condition, and adjust the angle of the lens and/or the image plane in order to focus on a scene that has an area of interest in a plane at an oblique angle to the optical axis. Conventionally, this is done by viewing the image projected on a ground glass screen in the camera, whilst adjusting the relative orientation of the lens and image plane to focus on the object plane of interest.
Although this principle has been known for many years, Scheimpflug cameras have remained manually operated and large format. One attempt to partially automate a Scheimpflug camera is described in patent document U.S. Pat. No. 4,564,277. This however, describes a camera suitable for professional film photography and which may have a ground glass screen for viewing an image. The process is only semi-automated, requiring a photographer to move an image plane perpendicular to the optical axis to at least two locations where different areas of the screen are in focus, and enter (x,y,z) position data for these locations into a calculator, which from the known focal length of the lens, can then calculate the correct orientation of the lens or image plane to meet the Scheimpflug condition. Such a camera and process are not sufficiently inexpensive, quick or convenient for a mass-market desktop imaging product.
It is an object of the present invention to provide an electronic camera that addresses these problems.
Accordingly, the invention provides an electronic, camera, comprising: a detector array; an objective lens arranged to direct optical radiation from an object plane onto the detector, the lens defining an optical axis of the camera, and the object plane being at an oblique angle to the optical axis; movement means to change the relative orientation of the detector with respect to the lens so that the detector and lens may be moved relatively toward or away from each other along the optical axis and also tilted with respect to each other with at least one degree of freedom; and focus detection means connected to the detector to detect when a portion of an image falling on the detector is in focus, characterized in that the camera includes a processor means to control the movement means according to the detected focus, the processor means bringing the image into focus on the detector by first changing the relative orientation of the lens and detector until a first portion of the image is in focus, and then holding said first portion in focus whilst continuing to change the relative orientation of the lens and detector until a second portion of the image is also in focus.
The term xe2x80x9clensxe2x80x9d as used herein is not restricted to a single lens element and includes lenses with compound optical elements.
The focus detection means may be any of the known automatic focusing techniques that are found in digital or film-based still or video cameras. These include range finding techniques based on the transmission and reception of reflected infrared pulses. Additional focus means may also be employed, for example those relying on reflected ultrasonic pulses. Other known techniques that involve some sort of statistical or frequency domain analysis of the detected image, are particularly appropriate when the object is imaged electronically, rather than with photographic film, as with video or digital still cameras.
Acoustic and, particularly, infrared range finders are very directional. It would be possible to fit three of these to the camera pointing in different directions within the field of view. The position of a plane is defined by the position of three non-collinear points. Knowing the position of the object plane, the required angle of the image (or lens) plane could then be calculated directly by 3-D geometry. With the simplification discussed above only two range finders would be needed.
The detector may be any type of electronic detector, in particular a two-dimensional CCD array or CMOS array.
Assuming a reasonably designed lens, whatever the position of the image or lens planes, the image is a perspective projection of the original object. Thus, any straight line on the object plane is also a straight line on the image plane. A consequence of this is that it is unnecessary to go through the procedure described in U.S. Pat. No. 4,564,277 to determine the position of the image plane and then, from this and a knowledge of the focal length, calculating the required positions of the lens and/or image planes using Scheimpflug""s rule. It is sufficient to adjust the focus directly. No calculation of the position of any of the planes is required. Neither is it necessary to know the focal length of the lens.
Preferably, the processor means brings the image into focus by first moving the lens and detector relatively towards or away from each other along the optical axis until the first portion of the detector is in focus, and then holding said first portion in focus whilst tilting with respect to each other the lens and detector until the second portion is also in focus.
When the image of an object in an object plane oblique to the optical axis comes into focus on the detector, the object, lens and detector satisfy the Scheimpflug condition. The lens may become mechanically complicated if it has to be correctly positioned both for a general or rough focus by moving in and out with respect to a camera body, as well as having to be tilted. Although it is possible to move the plane of the lens through a number of positions meeting the Scheimpflug condition until the position of best focus is found, it is preferred to leave the lens in a fixed position and move the detector to the image plane in order to meet the same condition, because this may be mechanically simpler and lends itself to be performed automatically.
Therefore the lens may be movable along the optical axis with respect to the camera body, with the detector being movable with respect to the body at least to tilt the detector with respect to the lens. Optionally, some small amount of general focus may be provided for by the detector by movement of the detector along the optical axis without tilting.
Alternatively, and particularly in the case where the operating focus distance is always within well-defined limits, say of the order of 0.2 m to 2 m in a desktop imaging application, the lens may be fixed relative to the body, with the detector being movable with respect to the body along the optical axis and to tilt the detector with respect to the lens.
In either case, the detector array will then be physically moved by the movement means, for example being mounted at corners on linear actuators that allow the detector to move backwards and forwards and also tilt with respect to the optical axis. If the camera is to be used in an application where essentially only one degree of tilt freedom is needed, for example if a document to be imaged will always be directly in front of the camera on a surface tilted relative to the optical axis about a line at right angles to this axis, then it need have only one degree of tilt freedom. A consequence of this simplification of camera motion is that there is a corresponding simplification in focusing motion, requiring only two degrees of freedom: linear movement and tilt in one direction. This results in simplification of processor control as well.
However, in most cases, it is envisaged that the camera may have to cope with an object tilted about a line not at right angles to the optical axis. Therefore, the detector and lens may be arranged so that these can be tilted relative to each other with two essentially orthogonal degrees of freedom. The processor means then brings the image into focus by first bringing the first and second portions into focus, and then holding said first and second portions in focus whilst tilting with respect to each other the lens and detector until a third portion of the image is also in focus.
In a second embodiment of the invention, an electronic camera comprises additionally: a second lens disposed between the objective lens and the detector and arranged to direct optical radiation from an image plane of the objective lens onto the detector, the second lens defining an internal optical axis of the camera that intersects the optical axis of the objective lens; and a rotation means to change the relative orientation of the second optical axis and the optical axis of the objective lens, the processor means being adapted to control the rotation means in order to improve detector keystone distortion in the image plane of the second lens.
The above term xe2x80x9cimprovexe2x80x9d means in this context either ameliorating or eliminating keystone distortion.
Surprisingly, it is in principle possible to eliminate keystone distortion whilst at the same time bringing the image of the tilted object into focus on the detector by satisfying a double Scheimpflug condition, one for each lens. In addition, this can be done automatically by the processor means, using the focus detection in the detector portions mentioned above, and knowledge of the focal length of the both lenses, and the separation between these lenses.
Specifically, the camera may comprise comprises memory means holding data representative of: the focal length of both the objective and second lenses, and the separation of the lenses when the axes of the lenses are aligned. The camera then comprises means to determine the relative orientation of the detector and the second lens when the portions of the image are in focus and to generate data representative of said orientation. Finally, the processor means can be arranged to calculate from said focal length, separation and orientation data a relative orientation of the objective lens, second lens and detector that will focus the object plane onto the detector whilst at the same time improving keystone distortion.
In the case where the object plane is tilted about a line at right angles to the objective optical axis, the rotation means may need only one degree of freedom. In general, however, the second optical axis may be rotated with respect to the first optical axis with two essentially orthogonal degrees of freedom.
The electronic camera described above may be used in a desktop imaging application, in place of a flatbed scanner. Such an imaging device may have a mount, for example a pole or bracket clampable to an edge of a desk, by which the camera may be mounted above the edge of the desk and directed down onto the desk in order to image a document on the desk.
A desktop imaging device positioned above and to one side of a work surface on a desk naturally takes advantage of the Scheimpflug condition because the desktop is inherently a flat plane.
Also according to the invention, there is provided a method of imaging an object, using an electronic camera comprising a detector array, an objective lens arranged to direct optical radiation from an object plane onto the detector, the lens defining an optical axis of the camera, movement means to change the relative orientation of the detector with respect to the lens so that these may be moved relatively toward or away from each other along the optical axis and also tilted with respect to each other with at least one degree of freedom, and focus detection means connected to the detector to detect when a portion of an image falling on the detector is in focus, the method comprising the first step of pointing the camera at the object so that the object plane is at an oblique angle to the optical axis, characterized in that the camera includes a processor means to control the movement means according to the detected focus, and in that the method comprises the additional steps of:
i) using the processor means to bring the image into focus on the detector by first changing the relative orientation of the lens and detector until a first portion of the image is in focus; and then
ii) holding said first portion in focus whilst continuing to change the relative orientation of the lens and detector until a second portion of the image is also in focus.
The method may also be adapted to the case where it is desired to improve keystone distortion in an electronic camera having a second lens, as mentioned above. Therefore, when the processor means is adapted to control the rotation means, the method comprises the step of:
iii) using the processor means to change the relative rotation of the second optical axis and the optical axis of the objective lens in order to improve detector keystone distortion in the image plane of the second lens.
When the camera comprises memory means, and means to determine the relative orientation of the detector as mentioned above, the method comprises the step of:
iv) storing in the memory means data representative of: the focal length of both the objective and second lenses, and the separation of the lenses when the axes of the lenses are aligned;
and after step ii) the steps of:
v) using the processor means to calculate from said focal length, separation and orientation data a desired relative orientation of the objective lens, second lens and detector; and
vi) using the processor means to change the relative orientation of the objective lens, second lens and detector in order to focus the object plane onto the detector whilst at the same time improving keystone distortion.